-
Open
-
Open
-
Open
-
Open
-
Open
-
Open
An Empirical Evaluation of the Rashomon Effect in Explainable Machine Learning. (arXiv:2306.15786v2 [cs.LG] UPDATED)
The Rashomon Effect describes the following phenomenon: for a given dataset
there may exist many models with equally good performance but with different
solution strategies. The Rashomon Effect has implications for Explainable
Machine Learning, especially for the comparability of explanations. We provide
a unified view on three different comparison scenarios and conduct a
quantitative evaluation across different datasets, models, attribution methods,
and metrics. We find that hyperparameter-tuning plays a role and that metric
selection matters. Our results provide empirical support for previously
anecdotal evidence and exhibit challenges for both scientists and
practitioners.
( 2
min )
Understanding Graph Neural Networks with Generalized Geometric Scattering Transforms. (arXiv:1911.06253v5 [stat.ML] UPDATED)
The scattering transform is a multilayered wavelet-based deep learning
architecture that acts as a model of convolutional neural networks. Recently,
several works have introduced generalizations of the scattering transform for
non-Euclidean settings such as graphs. Our work builds upon these constructions
by introducing windowed and non-windowed geometric scattering transforms for
graphs based upon a very general class of asymmetric wavelets. We show that
these asymmetric graph scattering transforms have many of the same theoretical
guarantees as their symmetric counterparts. As a result, the proposed
construction unifies and extends known theoretical results for many of the
existing graph scattering architectures. In doing so, this work helps bridge
the gap between geometric scattering and other graph neural networks by
introducing a large family of networks with provable stability and invariance
guarantees. These results lay the groundwork for future deep learning
architectures for graph-structured data that have learned filters and also
provably have desirable theoretical properties.
( 2
min )
SVDinsTN: An Integrated Method for Tensor Network Representation with Efficient Structure Search. (arXiv:2305.14912v2 [cs.LG] UPDATED)
Tensor network (TN) representation is a powerful technique for data analysis
and machine learning. It practically involves a challenging TN structure search
(TN-SS) problem, which aims to search for the optimal structure to achieve a
compact representation. Existing TN-SS methods mainly adopt a bi-level
optimization method that leads to excessive computational costs due to repeated
structure evaluations. To address this issue, we propose an efficient
integrated (single-level) method named SVD-inspired TN decomposition
(SVDinsTN), eliminating the need for repeated tedious structure evaluation. By
inserting a diagonal factor for each edge of the fully-connected TN, we
calculate TN cores and diagonal factors simultaneously, with factor sparsity
revealing the most compact TN structure. Experimental results on real-world
data demonstrate that SVDinsTN achieves approximately $10\sim{}10^3$ times
acceleration in runtime compared to the existing TN-SS methods while
maintaining a comparable level of representation ability.
( 2
min )
Learning Mixtures of Gaussians with Censored Data. (arXiv:2305.04127v2 [cs.LG] UPDATED)
We study the problem of learning mixtures of Gaussians with censored data.
Statistical learning with censored data is a classical problem, with numerous
practical applications, however, finite-sample guarantees for even simple
latent variable models such as Gaussian mixtures are missing. Formally, we are
given censored data from a mixture of univariate Gaussians $$
\sum_{i=1}^k w_i \mathcal{N}(\mu_i,\sigma^2), $$ i.e. the sample is observed
only if it lies inside a set $S$. The goal is to learn the weights $w_i$ and
the means $\mu_i$. We propose an algorithm that takes only
$\frac{1}{\varepsilon^{O(k)}}$ samples to estimate the weights $w_i$ and the
means $\mu_i$ within $\varepsilon$ error.
( 2
min )
Deep Learning for Energy Time-Series Analysis and Forecasting. (arXiv:2306.09129v2 [cs.LG] UPDATED)
Energy time-series analysis describes the process of analyzing past energy
observations and possibly external factors so as to predict the future.
Different tasks are involved in the general field of energy time-series
analysis and forecasting, with electric load demand forecasting, personalized
energy consumption forecasting, as well as renewable energy generation
forecasting being among the most common ones. Following the exceptional
performance of Deep Learning (DL) in a broad area of vision tasks, DL models
have successfully been utilized in time-series forecasting tasks. This paper
aims to provide insight into various DL methods geared towards improving the
performance in energy time-series forecasting tasks, with special emphasis in
Greek Energy Market, and equip the reader with the necessary knowledge to apply
these methods in practice.
( 2
min )
Orbit Classification of asteroids using implementation of radial Basis Function on Support Vector Machines. (arXiv:2306.17138v1 [astro-ph.EP])
This research paper focuses on the implementation of radial Basis Function
(RBF) Support Vector Machines (SVM) for classifying asteroid orbits. Asteroids
are important astronomical objects, and their orbits play a crucial role in
understanding the dynamics of the solar system. The International Astronomical
Union maintains data archives that provide a playground to experiment with
various machine-learning techniques. In this study, we explore the application
of RBF SVM algorithm to classify asteroids. The results show that the RBF SVM
algorithm provides a good efficiency and accuracy to the dataset. We also
analyze the impact of various parameters on the performance of the RBF SVM
algorithm and present the optimal parameter settings. Our study highlights the
importance of using machine learning techniques for classifying asteroid orbits
and the effectiveness of the RBF SVM algorithm in this regard.
( 2
min )
An image segmentation algorithm based on multi-scale feature pyramid network. (arXiv:2305.10631v4 [eess.IV] UPDATED)
Medical image segmentation is particularly critical as a prerequisite for
relevant quantitative analysis in the treatment of clinical diseases. For
example, in clinical cervical cancer radiotherapy, after acquiring subabdominal
MRI images, a fast and accurate image segmentation of organs and tumors in MRI
images can optimize the clinical radiotherapy process, whereas traditional
approaches use manual annotation by specialist doctors, which is time-consuming
and laborious, therefore, automatic organ segmentation of subabdominal MRI
images is a valuable research topic.
( 2
min )
Non-Convex Optimizations for Machine Learning with Theoretical Guarantee: Robust Matrix Completion and Neural Network Learning. (arXiv:2306.16557v1 [cs.LG])
Despite the recent development in machine learning, most learning systems are
still under the concept of "black box", where the performance cannot be
understood and derived. With the rise of safety and privacy concerns in public,
designing an explainable learning system has become a new trend in machine
learning. In general, many machine learning problems are formulated as
minimizing (or maximizing) some loss function. Since real data are most likely
generated from non-linear models, the loss function is non-convex in general.
Unlike the convex optimization problem, gradient descent algorithms will be
trapped in spurious local minima in solving non-convex optimization. Therefore,
it is challenging to provide explainable algorithms when studying non-convex
optimization problems. In this thesis, two popular non-convex problems are
studied: (1) low-rank matrix completion and (2) neural network learning.
( 2
min )
Forecasting of the development of a partially-observed dynamical time series with the aid of time-invariance and linearity. (arXiv:2306.16593v1 [stat.ME])
A dynamical system produces a dependent multivariate sequence called
dynamical time series, developed with an evolution function. As variables in
the dynamical time series at the current time-point usually depend on the whole
variables in the previous time-point, existing studies forecast the variables
at the future time-point by estimating the evolution function. However, some
variables in the dynamical time-series are missing in some practical
situations. In this study, we propose an autoregressive with slack time series
(ARS) model. ARS model involves the simultaneous estimation of the evolution
function and the underlying missing variables as a slack time series, with the
aid of the time-invariance and linearity of the dynamical system. This study
empirically demonstrates the effectiveness of the proposed ARS model.
( 2
min )
Neural networks can detect model-free static arbitrage strategies. (arXiv:2306.16422v1 [q-fin.CP])
In this paper we demonstrate both theoretically as well as numerically that
neural networks can detect model-free static arbitrage opportunities whenever
the market admits some. Due to the use of neural networks, our method can be
applied to financial markets with a high number of traded securities and
ensures almost immediate execution of the corresponding trading strategies. To
demonstrate its tractability, effectiveness, and robustness we provide examples
using real financial data. From a technical point of view, we prove that a
single neural network can approximately solve a class of convex semi-infinite
programs, which is the key result in order to derive our theoretical results
that neural networks can detect model-free static arbitrage strategies whenever
the financial market admits such opportunities.
( 2
min )
Complex-valued Adaptive System Identification via Low-Rank Tensor Decomposition. (arXiv:2306.16428v1 [cs.LG])
Machine learning (ML) and tensor-based methods have been of significant
interest for the scientific community for the last few decades. In a previous
work we presented a novel tensor-based system identification framework to ease
the computational burden of tensor-only architectures while still being able to
achieve exceptionally good performance. However, the derived approach only
allows to process real-valued problems and is therefore not directly applicable
on a wide range of signal processing and communications problems, which often
deal with complex-valued systems. In this work we therefore derive two new
architectures to allow the processing of complex-valued signals, and show that
these extensions are able to surpass the trivial, complex-valued extension of
the original architecture in terms of performance, while only requiring a
slight overhead in computational resources to allow for complex-valued
operations.
( 2
min )
-
Open
Numerical Data Imputation for Multimodal Data Sets: A Probabilistic Nearest-Neighbor Kernel Density Approach. (arXiv:2306.16906v1 [stat.ML])
Numerical data imputation algorithms replace missing values by estimates to
leverage incomplete data sets. Current imputation methods seek to minimize the
error between the unobserved ground truth and the imputed values. But this
strategy can create artifacts leading to poor imputation in the presence of
multimodal or complex distributions. To tackle this problem, we introduce the
$k$NN$\times$KDE algorithm: a data imputation method combining nearest neighbor
estimation ($k$NN) and density estimation with Gaussian kernels (KDE). We
compare our method with previous data imputation methods using artificial and
real-world data with different data missing scenarios and various data missing
rates, and show that our method can cope with complex original data structure,
yields lower data imputation errors, and provides probabilistic estimates with
higher likelihood than current methods. We release the code in open-source for
the community: https://github.com/DeltaFloflo/knnxkde
( 2
min )
Forecasting of the development of a partially-observed dynamical time series with the aid of time-invariance and linearity. (arXiv:2306.16593v1 [stat.ME])
A dynamical system produces a dependent multivariate sequence called
dynamical time series, developed with an evolution function. As variables in
the dynamical time series at the current time-point usually depend on the whole
variables in the previous time-point, existing studies forecast the variables
at the future time-point by estimating the evolution function. However, some
variables in the dynamical time-series are missing in some practical
situations. In this study, we propose an autoregressive with slack time series
(ARS) model. ARS model involves the simultaneous estimation of the evolution
function and the underlying missing variables as a slack time series, with the
aid of the time-invariance and linearity of the dynamical system. This study
empirically demonstrates the effectiveness of the proposed ARS model.
( 2
min )
Learning Mixtures of Gaussians with Censored Data. (arXiv:2305.04127v2 [cs.LG] UPDATED)
We study the problem of learning mixtures of Gaussians with censored data.
Statistical learning with censored data is a classical problem, with numerous
practical applications, however, finite-sample guarantees for even simple
latent variable models such as Gaussian mixtures are missing. Formally, we are
given censored data from a mixture of univariate Gaussians $$
\sum_{i=1}^k w_i \mathcal{N}(\mu_i,\sigma^2), $$ i.e. the sample is observed
only if it lies inside a set $S$. The goal is to learn the weights $w_i$ and
the means $\mu_i$. We propose an algorithm that takes only
$\frac{1}{\varepsilon^{O(k)}}$ samples to estimate the weights $w_i$ and the
means $\mu_i$ within $\varepsilon$ error.
( 2
min )
Medoid splits for efficient random forests in metric spaces. (arXiv:2306.17031v1 [stat.ME])
This paper revisits an adaptation of the random forest algorithm for
Fr\'echet regression, addressing the challenge of regression in the context of
random objects in metric spaces. Recognizing the limitations of previous
approaches, we introduce a new splitting rule that circumvents the
computationally expensive operation of Fr\'echet means by substituting with a
medoid-based approach. We validate this approach by demonstrating its
asymptotic equivalence to Fr\'echet mean-based procedures and establish the
consistency of the associated regression estimator. The paper provides a sound
theoretical framework and a more efficient computational approach to Fr\'echet
regression, broadening its application to non-standard data types and complex
use cases.
( 2
min )
Understanding Graph Neural Networks with Generalized Geometric Scattering Transforms. (arXiv:1911.06253v5 [stat.ML] UPDATED)
The scattering transform is a multilayered wavelet-based deep learning
architecture that acts as a model of convolutional neural networks. Recently,
several works have introduced generalizations of the scattering transform for
non-Euclidean settings such as graphs. Our work builds upon these constructions
by introducing windowed and non-windowed geometric scattering transforms for
graphs based upon a very general class of asymmetric wavelets. We show that
these asymmetric graph scattering transforms have many of the same theoretical
guarantees as their symmetric counterparts. As a result, the proposed
construction unifies and extends known theoretical results for many of the
existing graph scattering architectures. In doing so, this work helps bridge
the gap between geometric scattering and other graph neural networks by
introducing a large family of networks with provable stability and invariance
guarantees. These results lay the groundwork for future deep learning
architectures for graph-structured data that have learned filters and also
provably have desirable theoretical properties.
( 2
min )
Gradient flows on graphons: existence, convergence, continuity equations. (arXiv:2111.09459v3 [math.PR] UPDATED)
Wasserstein gradient flows on probability measures have found a host of
applications in various optimization problems. They typically arise as the
continuum limit of exchangeable particle systems evolving by some mean-field
interaction involving a gradient-type potential. However, in many problems,
such as in multi-layer neural networks, the so-called particles are edge
weights on large graphs whose nodes are exchangeable. Such large graphs are
known to converge to continuum limits called graphons as their size grow to
infinity. We show that the Euclidean gradient flow of a suitable function of
the edge-weights converges to a novel continuum limit given by a curve on the
space of graphons that can be appropriately described as a gradient flow or,
more technically, a curve of maximal slope. Several natural functions on
graphons, such as homomorphism functions and the scalar entropy, are covered by
our set-up, and the examples have been worked out in detail.
( 2
min )
Trajectory Poisson multi-Bernoulli mixture filter for traffic monitoring using a drone. (arXiv:2306.16890v1 [cs.CV])
This paper proposes a multi-object tracking (MOT) algorithm for traffic
monitoring using a drone equipped with optical and thermal cameras. Object
detections on the images are obtained using a neural network for each type of
camera. The cameras are modelled as direction-of-arrival (DOA) sensors. Each
DOA detection follows a von-Mises Fisher distribution, whose mean direction is
obtain by projecting a vehicle position on the ground to the camera. We then
use the trajectory Poisson multi-Bernoulli mixture filter (TPMBM), which is a
Bayesian MOT algorithm, to optimally estimate the set of vehicle trajectories.
We have also developed a parameter estimation algorithm for the measurement
model. We have tested the accuracy of the resulting TPMBM filter in synthetic
and experimental data sets.
( 2
min )
The Local Approach to Causal Inference under Network Interference. (arXiv:2105.03810v4 [econ.EM] UPDATED)
We propose a new nonparametric modeling framework for causal inference when
outcomes depend on how agents are linked in a social or economic network. Such
network interference describes a large literature on treatment spillovers,
social interactions, social learning, information diffusion, disease and
financial contagion, social capital formation, and more. Our approach works by
first characterizing how an agent is linked in the network using the
configuration of other agents and connections nearby as measured by path
distance. The impact of a policy or treatment assignment is then learned by
pooling outcome data across similarly configured agents. We demonstrate the
approach by proposing an asymptotically valid test for the hypothesis of policy
irrelevance/no treatment effects and bounding the mean-squared error of a
k-nearest-neighbor estimator for the average or distributional policy
effect/treatment response.
( 2
min )
Causal inference for the expected number of recurrent events in the presence of a terminal event. (arXiv:2306.16571v1 [stat.ME])
We study causal inference and efficient estimation for the expected number of
recurrent events in the presence of a terminal event. We define our estimand as
the vector comprising both the expected number of recurrent events and the
failure survival function evaluated along a sequence of landmark times. We
identify the estimand in the presence of right-censoring and causal selection
as an observed data functional under coarsening at random, derive the
nonparametric efficiency bound, and propose a multiply-robust estimator that
achieves the bound and permits nonparametric estimation of nuisance parameters.
Throughout, no absolute continuity assumption is made on the underlying
probability distributions of failure, censoring, or the observed data.
Additionally, we derive the class of influence functions when the coarsening
distribution is known and review how published estimators may belong to the
class. Along the way, we highlight some interesting inconsistencies in the
causal lifetime analysis literature.
( 2
min )
Neural networks can detect model-free static arbitrage strategies. (arXiv:2306.16422v1 [q-fin.CP])
In this paper we demonstrate both theoretically as well as numerically that
neural networks can detect model-free static arbitrage opportunities whenever
the market admits some. Due to the use of neural networks, our method can be
applied to financial markets with a high number of traded securities and
ensures almost immediate execution of the corresponding trading strategies. To
demonstrate its tractability, effectiveness, and robustness we provide examples
using real financial data. From a technical point of view, we prove that a
single neural network can approximately solve a class of convex semi-infinite
programs, which is the key result in order to derive our theoretical results
that neural networks can detect model-free static arbitrage strategies whenever
the financial market admits such opportunities.
( 2
min )
